1. Field of the Invention
This invention relates generally to data analysis and more particularly to a system and method for analyzing and aggregating an unlimited amount of data and delivering customizable reports generally used for business intelligence purposes.
2. Description of the Related Art
Vast amounts of data are available for corporations concerning their customers. A business has numerous contact points with customers including but not limited to the Internet, Interactive Voice Response (IVR) systems, private company databases, and Enterprise Resource Planning (ERP) systems. Each of these customer contact points or data sources contain data capable of being mined for business intelligence purposes.
The Internet has allowed unlimited access for customers to a company""s web site. This unlimited access creates a wealth of information in the form of web log files. Companies can use the web log files to extract information concerning the customers use of the web site.
An organization may have a phone system capable of Interactive Voice Response that customers routinely access. Data stored in the IVR data files can be presented to provide a profile of a customer""s use of the IVR system. In addition, private company databases also contain data files that can be mined for business intelligence purposes.
Typically a business will have in-house databases containing accounting, financial and sales data. These in-house databases are commonly referred to as ERP systems and are a valuable source of financial data.
It can be appreciated that there are a large number of sources containing data that can be aggregated by businesses to analyze customer interactions. The data contained in each of the above referenced data sources consists of various data formats. Under current practices, analyzing and generating meaningful reports from the various data sources is an expensive and time-consuming process.
In order to effectively extract and organize the data, a company often times requires professional data handlers such as system administrators, database administrators, programmers and business analysts to analyze each data file, to understand the data type and to organize the relevant information. The professional data handlers then take the relevant data and place it in a uniform format (e.g., tables database formats, spreadsheet formats, etc.), thereby generating a static report summarizing the information associated with the business parameters of interest and customer interaction with various forms of data. For a large enterprise this effort can take teams of professionals whose sole focus is to routinely analyze this data and continually format the data to generate the static reports. For example, FIG. 1 displays a flowchart of the current process employed by industry. The various data files (DF1, DF2, DF3, and DFn) represent the vast amount of data available to an organization. Typical data files include web log files 102a, phone system data 102b, private company databases 102c and in-house operations data 102n. As described above, the data contained in each of the data files are of various formats. Each of the data files must be looked at to understand the data type and organize the data 104a, 104b, 104c and 104n. This is performed manually by professional data handlers 106. The professional data handlers 106 manipulate the data so that the data can be presented in a static report 108 that is dated.
In addition, the manual data handling described above places the information extracted from each individual data file into a separate database, each database having a common format. Therefore, retrieval of the information is from a number of databases and not from a central location, thereby resulting in inefficiencies in manipulating, storing, and then presenting the requested data. It is also important to note that such pre-processing of the data can take so much time that the data that is finally presented is actually dated. For this reason alone, the resultant data is oftentimes rendered useless for making time sensitive decisions regarding important business actions in response to customer activity. Much of the aforementioned problems with data manipulation and business intelligence services arise because of the lack of any software capable of extracting, organizing and uniformly formatting the required data from the various customer contact points, without excessive human interaction by computer programmers and other professionals. Moreover, report updates require the entire process to be repeated, thereby incurring a high cost and not allowing for the generation of true on-demand reports. Accordingly, these services tend to be limited to large organizations that can afford the costs of processing the data from the various data sources.
As a result, there is a need for a solution to solve the problems of the prior art to effectively extract, organize and uniformly format the customer interaction data from various customer contact points and other business data sources. There is also a need for methods that enable accurate, efficient and timely presentation of the uniformly formatted data in the form of user requested reports.
Broadly speaking, the present invention fills these needs by providing a flexible dynamic and efficient method for acquiring and presenting data from various sources to be used for business intelligence purposes. It should be appreciated that the present invention can be implemented in numerous ways, including as a process, an apparatus, a system, computer readable media, or a device. Several inventive embodiments of the present invention are described below.
In one embodiment, a method for acquiring and presenting data for business analysis is provided. In this embodiment, the method identifies a set of data sources to be accessed, accesses the data sources and then acquires raw data from each of the data sources. Then the acquired raw data is stored into raw data type specific compartments, such that each raw data type specific compartment is associated with a set of data sources. Next the acquired raw data is pre-processed to ensure data integrity. Then the acquired raw data from each raw data type specific compartment is transferred to processed data type specific compartments defining processed data. The transfer process is configured to detect a data type of each raw data type specific compartment and load data components of each raw data type specific compartment into respective processed data type specific compartments based on the detected data type. The transfer process further includes converting the data types of each raw data type specific compartment into a single data type so that each processed data type specific compartment has the same single data type. Then the processed data from each processed data type specific compartment is transferred to a temporary database where display rules are applied. The display rules are configured to define a format and content of a requested report. Next the requested report is outputted by referencing the temporary database and acquiring data elements from the temporary database using the display rules.
In another embodiment, a method for acquiring and transforming data for business analysis is disclosed. In this embodiment, the method identifies a set of data sources to be accessed. Then the raw data from each of the accessed set of data sources is loaded to raw data type specific locations. Next the integrity of the acquired raw data is verified. Then the acquired raw data from each of the raw data type specific locations is transformed to a uniform data type and then stored as processed data into processed data type specific locations.
In yet another embodiment, a method for acquiring and transforming data for business analysis is provided. In this embodiment, the method identifies a set of data sources to be accessed for a user, loads the raw data from each of the accessed set of data sources then verifies the integrity of the acquired raw data. Next the acquired raw data is transformed to processed data. The transformation process includes transferring the acquired raw data to a temporary database, converting the acquired raw data to a uniform format then applying processing rules to the converted raw data where the processing rules are configured to define processed data. Next the processed data is stored in a processed data type specific location from the temporary database. Next the processed data is transferred to a temporary database and display rules are applied where the display rules are configured to define a format and content of a requested report.
In still another embodiment, a method for acquiring and transforming data for business analysis is disclosed. In this embodiment, the method identifies a set of data sources to be accessed for a user, loads the raw data from each of the accessed set of data sources then verifies the integrity of the acquired raw data. Next the acquired raw data is transformed to processed data. The transformation process includes transferring the acquired raw data to a temporary database where the transfer is performed by a parallel memory map, converting the acquired raw data to a uniform format and then applying processing rules to the converted raw data where the processing rules are configured to define processed data. Next the processed data is stored in a processed data type specific location from the temporary database. Next the processed data is transferred to a temporary database and display rules are applied where the display rules are configured to define a format and content of a requested report.
In another embodiment, a method for retrieving, organizing and presenting data for business intelligence purposes is disclosed. In this embodiment the method locates a set of data sources containing raw data files. Next a time and frequency to access each of the set of data sources is set. Then the raw data files from the set of data sources is retrieved with the raw data files containing raw data. Next each of the retrieved raw data files is stored in a storage file. Then each of the retrieved raw data files is processed, where the processing includes loading each of the retrieved raw data files in a loader, where the loader contains a set of processing rules, and applying the set of processing rules to convert each of the retrieved raw data files to a processed data file having a uniform format. Next each of the processed data files is stored in the storage file. Then each of the processed data files are transferred to a database. Next the database is accessed in response to a customer report request. Next a report is presented in a desired format to the customer.
In yet another embodiment, a computer readable media having program instructions for acquiring and transforming data for business analysis is provided. One exemplary computer readable media includes program instructions for identifying a set of data sources to be accessed for a user. In addition, the computer readable media includes program instructions for loading raw data from each of the accessed set of data sources to raw data type specific locations. Additionally, the computer readable media includes program instructions for verifying the integrity of acquired raw data. In addition, the computer readable media includes program instructions for transforming the acquired raw data of each of the raw data type specific locations to a uniform data type, where the transformed acquired raw data is stored as processed data into processed data type specific locations.
The advantages of the present invention are numerous. Most notably, the methods for acquiring and presenting data for business analysis purposes efficiently extract, organize and present the data as requested by the customer. The conversion of the data to a uniform format allows for the centralization of the data thereby eliminating the need to continually access various data sources to provide requested reports. By automating the acquisition of the data through a data acquisition engine and storing incremental updates in a uniform format, the customer can be provided ad-hoc dynamic reports in real-time (e.g., in web database formats), thereby allowing for time sensitive and mission critical business decision to be made with all the current data at the executives fingertips. In addition, the utilization of temporary databases combined with parallel memory mapping transfers, minimizes the time required to retrieve and organize the data for presentation, in essence eliminating the need for teams of personnel to manually manipulate the data and both build and maintain costly, complex database systems. Furthermore, due to the uniform centralized format of the acquired data, the presentation of the data can be modified and re-presented in a fraction of the time as is currently available under the prior art. Finally, the presentation of the data can be customized to present a multi-dimensional display of the real world raw data enabling the customer a 360xc2x0 view of what is happening with an organization.
Other aspects and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.